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2. Cross-References to Related Applic ations. If Any. 

This application is a Continuation of U. S. Serial No. 09/545,024 filed 04/07/2000; which 
is a Continuation-In-Part of U.S. Serial No. 09/408,213 filed 09/27/1999; which is a 
Continuation of U.S. Serial No. 08/892,919 filed 07/15/1997, and claims benefit of U.S. 
Provisional Application No. 60/021,980 filed 07/18/1996. Each of these related applications are 
incorporated by reference, herein. 

3. Sf«ti>ment as to Rights to Invent i nn. Made Under Federally Sponsored Research 
and Development. If Any. 

This patent is not based upon any federally sponsored research and development. 

4. Background of Invention . 

A. Field of Invention . 

The present invention is in the field of systems and articles of manufacture to 
administer complex, heterogeneous networked computer systems. 

B. Related Background Art 

Prior art systems were deficient generally for two reasons: first, limitations inherent in 
available user interfaces, and second, absence of open-architecture, integrated systems 
effectively to manage and administer heterogeneous platforms using diverse operating systems 
for many different applications, including information technology and business management 
administration and to isolate views of specific business and management interests. 

Prior art graphical user interfaces of administrative systems attempted to administer 
multi-unit computer networks by causing any of the four categories of information to appear on 
the computer monitor being used by the system administrator. 
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1. Lists, two-dimensional and on scrollable screens, typically using a windows 
program manager with many subUsts showing printers, operating systems, physical sights, etc. 

2. Tree diagrams showing the hierarchical relationships of the network system such 
as by showing the various geographical locations, the number of buildings at a location, the 
number of computers at each location, and the peripheral equipment associated with each 
computer and the systems being operated on each computer. 

3. A hierarchical structure using folders and icons with each folder being a list of 
icons and with each icon by its color indicating the status of each unit. 

4. Diagrams, with icons, of the various systems in a hierarchy. 

Each of the displayed categories of information works well but with critical limitations. 
For example, the use of two dimensional lists is definitely limited by the number of units: as the 
number increases, the lists become effectively umnanageable by the user. Further, the 
hierarchical systems can allow increased navigating ability but are again limited to a few 
thousand devices and by the fact that the only relationships that can be displayed are those withm 
the hierarchy. TTiis is Umited typically because the tree structure is based on a single hierarchy; 
for example, it may be organized geographically and this will not allow display of units m 
multiple geographical locations that are a part of a particular business interest. The hierarchical 
systems also have the shortcoming that limited status-indicating information may be displayed m 
the available space. Even when using the folders-icon system, although multiple hierarchies can 
be displayed, the user tends to be confused or is provided incomplete information by the limited 
amount of data that can be provided. Managing the user interface itself becomes a bigger 
concern than managing the computer network. Although through certain enhancements, the tree 
diagram/map system can improve on its effectiveness, such as by showing a transmission line as 
green if it is functioning and red if it is not. it and the other prior art interface systems are still 

limited to several thousand units. 

In the case of all of the aforementioned prior art systems discussed above, none can be 
effectively used in the modem environment in which it is not uncommon to have 10,000 
computer devices to more than 100,000 such devices in a networked system. Further, the prior 
art systems limit the scope of the responsibility of the systems administrator. As the number of 
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units within a network system increases, the number of physical and logical relationships 
between the systems responsible for the various functions mcreases exponentially so it becomes 
very difficult, if not impossible, to manage the network system. Even trained professionals 
cannot deal with the enormous numbers of relationships that must be monitored and managed in 
the complex systems. With prior art systems administration interfaces, panel design, PF keys, 
and screen clutter prohibit the intuitive navigation that enables effective systems and enterprise 
management. 

The second broad category of deficiencies in the prior art relate to the absence of 
manageable systems for networks comprised of widely diverse hardware platforms and even 
more widely diverse software systems and specific application programs. For some time, as 
computer networks became more complex, systems administrators have needed the ability to 
have a view of the network that identifies and presents for viewing the units or assets that 
fimction in support of a particular application, and also to have the ability to apply systems 
management fimctions (asset utilization, alarms, software distribution, etc.) to manage the 
particular application. Prior to the present invention, a systems administrator would have to set 
up different systems for different platforms and appUcations: e.g., an administrator may need to 
set up Sun NetManager or Open-View or IBM's NetView to run LANs, then set up a different set 
of systems management tools for each of the other platforms in a user's enterprise-e.g., a system 
to track activities on ASMOOs; another administi-ation system may be needed for a UNIX host 
and server systems (and something different for each different UNIX OS, if there is more than 
one in a user's network). Further, mainfirame systems tools for security, backup, scheduling, etc.; 
plus software distribution tools, desktop asset management tools, help desk and trouble-ticketing 
tools all had to be separately provided, and their compatibility constantly was problematic. 

In the prior art, there has not been a system or apparatus that, on a single console, 

effectively and in combination: 

1. Uses 3-D virtual reality to map complex systems-business or Information 
Technology~to an intuitive and effective interface; 

2. Maps systems management tasks to business fimctions, not to system hardware or 
software; 
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3. Achieves end-to-end comprehensive integrated systems and network management 
of all elements of an IT network from a single or several points of control; 

4. Allows business process management of financial, manufacturing, distribution, 
systems, and network applications using a real world interface; and 

5. Brings functionally robust management tools to client/server systems. 

The need for a system to accomplish these objectives was the result of certain historical 
developments that resulted in many users having widely diverse computer systems. In the early 
years of computing, mainframe computers were widely used. The advent of chent/server 
systems brought a new dimension to systems management. Multiple computers, from a simple 
one client/one server enviromnent, to a complex array of different computers from different 
manufacturers supporting large and complex dient/server appUcations using a wide variety of 
software systems must be administered as if they were one interoperable system. 

In large networks, with hundreds, or even thousands, of workstations and dozens of 
servers, administration and management of the individual workstations is a very substantial task. 
TTie adminisfration can include workstation configuration control, system security, workstation 
fault correction, application monitoring for software license compUance, software application 
distribution, software version control, and customization of user enviromnent. In such large 
networks, administration became time-consuming and tedious because the system administration 
was in the same physical location as the workstation. Since these workstations are typically 
spread over a large areas such as a large, multi-story building, multiple cities, and even multiple 
countries, a significant amount of time and effort was spent in traveling between workstations to 

perform management tasks. 

In the prior art, the focus of system management was on network equipment and systems. 
See, Stafford, "Application Management - Client-Server's Missing Link," BaOnsiness. 
February 1, 1996, Volume 12, No. 2, p. 133. The prior art had developed infrastructure that 
support the users' key assets: enterprise dient-server applications and the data within them. 
However, client-server application management was non-existent. CUent-server users could only 
inefficiently, if at all. account for the assets within the system, determine what applications were 
on their networks, assess how those applications were performing, identify failures occurring in 
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hardware or software assets, and then diagnose and correct Mts. In part, because of these 
difficulties, planning for network growth also was a task that was difficult at best. Pnor to the 
invention described herein, there was no built-in way, efficiently, to get this information ,n a 

distributed application environment. 

Unsuccessfl.1 attempts have been made to develop an end-to-end solution to provide real 
time information about application health, administtation, service level and performance. 
Application health typically encompassed queues, process states, interrupts and network n-affic. 
Two dimensional lists of assets, presented on a monitor, were utili^. Using these lists, systems 
administrators dealt with application control issues, such as start/stop, user auihenticanon and 
load balancing. Service level and performance includes response time, trend analysis, threshold 
alert and predictive analysis. Failures were hard to trace when the application management was 
not part of a sy^em/network management scheme. Typically, businesses reported that a 
significant per^tage of client-server trouble reports were attributed to apphcation software. 
Mainftame systems had embedded, centraUzed application monitoring facilities. However, m 
distributed enviromnents, following the data flow is a complex task, since application and data 
go through many steps. Wore, there has been a long felt need for a system which could 
capture and act upon information about the behavior of all the applications nmmng on a 
networked system that included client-server systems. 

Developers have attempted to create a system to monitor a client-server network m tts 
entirety Conventional network management solutions stabilized the infrastmcmre fltat support 
the user's key assets, enten.rise client-server applications and the data within them. Then a 
second generation of development products we« developed which attempted to monitor a chent- 
server network in its entirety, across heterogeneous platforms, ftom a single console. These 
systems monitor certain functions such as CPU time, input/output and disk space and also 
perform and monitors security for the enterprise. They provided such fimctions as sendmg alert- 
or-perfonn-the-task sisals to enforce enterprise-wide poUcies for such thmgs as network 

performance and security access. 

However, as systems became larger, more widespread, and more heterogeneous, pnor to 
the present invention, there has been no acceptable method for a manager firlly to comprehend 



{M-A1079.DOC;1} 



5 



10 



15 



20 



25 



either the network system or the assets relevant to a particular malfunctioning subsystem. 
Conventional human interfaces, such as "trees" which displayed the structure of the network in 
text form, or simple icons, which represented parts of the system in two-dimensional form, were 
inadequate to provide a real-time system overview, or subsystem overview to allow the 
administrator to envision a system, its malfunction, and the corrective action needed. 

Thus, prior to the present invention, there has been no application management system 
and process.'which would provide an understandable, yet comprehensive, system-wide overview 
of the network, or of a subpart of the network. The present invention relates to a method and 
apparatus of providing a three dimensional, animated overview and system to monitor and 
troubleshoot even the most complex dient-server system. Also, prior to the present inventions, 
there has been no client-server administration system which not only can monitor an individual 
resource or specific platform, but also can provide an effective comiection between specific 
business operations and enterprise information technology management. 

The present invention achieves a broad reach of hardware platform integration across 
heterogeneous networks and applications. This allows the present invention to manage business 
processes and production activities such as by detecting a potential inventory shortage and 
sending out a rush order to the appropriate supplier. 

In many applications, including network management, modeling, web site design and 
project management, user interfaces can be based on graph diagrams. These diagrams show 
icons or shapes interconnected with lines. To convey more information about the objects and 
comiections. both may be amiotated with text and numbers, or drawn with different shapes, 

icons, colors or animated effects. 

It is also common that the objects in such diagrams, and sometimes the comiections as 
well, may contain further structures. The contents of an element in the diagram may be 
represented as another diagram of the same type, or in some other form, including other types of 
diagrams, property sheets or text. The most common type of navigation in user interfaces based 
on this concept is opening the component to see its contents. For example, by double-clickmg 
with the mouse, selecting a menu item or other similar action, the user replaces the current view 
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wi* a„o«,er one. TT.e new diagram may replace ttre current one in U,e prop's window, or 

may open another window. 

However, .his common user interface approach has several disadvantages. TTre .dd n 

ttansition tarn one dia^am to another has the effect of losing the context for the ^ the 
Zltsintheconta^eddiagramhavenovisihierelationshiptotheelementsofthecontam^^^ 

struct^. It is also psychologically Jarring, and interrupts the wor. ilow. Father. « enforces a 
hierarchical structure among the graphs that is not always sigmficant. 

Otoer user interfaces have used the corKcpt of continuous zoommg to reflect su h 
contaimnent structures. Icons are displayed on a virtual desl^op. and the us. c. se— 
^„m and pan on this desktop. As the user zooms m ^d the icons hecome larger m the us« 
Lace, L eternal strucmre appears, in the form of other icon, text or other m^^rmaho. 
The user interface permit indefirite zooming, as long as there is more informafon contamed m a 

visible element. . 

While such systems have several usahiUty advantages, they have not been able to 
represent me more complex smrctures that require graph dia^ams. with interconnected nodes. 
t.hefie.dofnetwor.management.thecommontechni<,ues for visualizingthe structure 

of .he network are nested 2-D diagrams. 3-D visuaUzation, tree controls and to new 
conUnuously zooming infinite graph diagram. Each of these techmques has advantages, but none 
Tgood a. Idling one particular problem: foUow-ngatrailofrelationshipsinave^l^^^^ 

biy graph. T1.e problem is common, and is charaCeHzed by a rapid fan-out of Imks. ^es^ 
Is m!y r epresen. phy»ica. network links, logical network links at varrous levels of a network 
;l or Jlogical dependency relationships that drive Quality of Service an.ys,s, .mpact 
analysis and root cause analysis. 

,f all of these links are displayed in a conventional static diagram, the dra^ rs rap dl 
overwhehned by tt,e number of lirJcs. Various prior-art visualization techniques attempt to deal 
with the problem through nesting, filtering or scrolling, bu, none is very successM. 

L hyperbolic tree is a well-known technique for visuaUzing directed graphs. It riders 

tt,e diagram as a straight-forward expanding tree, and solves the bushiness problem by rend^mg 
U,egrarhonahyperbolicsurfa..ll.edia^appearsto«>euserasifit.sdrawnonthe surface 
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of a sphere: as nodes get fWcr away torn fte ce„.er. ftey gC smaller and evenmally disappear 

over the horizon. 

5 Summary of Tnvention. 

A w„rM interface. The present invention is a system and apparatus for 

S visnali^g the components of a computer netwo* system as a realistic *ree^mensiona. 
environment for the purposes of systems and network management. The .hr^tmenstonal 
rendering called the "Real World Interface," by using "Virma, Reality" technology, shows 
computer systems, printers, network routers and other devices with the. ne^ork 
interconnections, in a realistic or stylized environment symbolizing a geo^hrc reg,o„ Uk a 
.0 country, region or city, togedter with huildinp. Tlte user of the system can trave » *e 
environment, using various interaction devices, and directly select devces for mampula«on. The 
useful, practical applicaUon of the present invention is to allow the administratton of systems 
comprising 10,000 units or more efliciently. by displaying in virtud reality on a computer 
monitor the relevant portions of a computer network, thus allowing dre use to be mtu...ve as tf 
15 physically present at numerous remote locations. 

Further the present invention allows the user to visualize all the informafon known to a 
distributed, multifaceted database, and to provide an ove^ew of all the data, by use o 
comprehensive, manageable, intuitive views that relate to practical business issues, ^e p«sent 
invention also includes a real world interface which uses automatic piloting or alt^atively. 
.0 manual piloting for traversing the networked topography Fast pathing and color coded al^ 
allow flte user to detennine precisely which re^urce is experiencmg a problem. Users can tt,en 
drill down to any node and access management tetions to resolve the problem or adrntmster 
fte system. TTte present invention exceeds the desi^ goals of prior art systems and mteri^ac^. 
However, the present invention offers a choice of user interfaces including tree views and two 
.S dimensional map views. AH of these user interfaces offer a hi^ degree of user defined 
customization and filtering capabilities including the ability to create business process vrews. 

Such views and visual aids allow a systems administrator to maximtze use of hts or her 
intuitive, communicative, and diagnostic skills in applying such diagnostic and corrective 
systems to address a malfcnction in hardware, firmware, or soflware. Business interest vews 
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filter the views ,o isolate specific business interests, such as management inventory or payroll, 
and then to present virtual reality views, allowing an admimstrator of a networlced computer 
systemtoreviewandmanagethespecificassetstlmtrelatetothatbusinessinterest 

B -^- r---'"' ^" ™' 

interface of the present invention provides a r.l-time 3-D view of all the assets in a networked 
computer system, ftom the ^obal network, to the computers i„ each area, to their processors ^ 
drives, down to abstract objects such as databases, applications and «g processes. V.. 
present invention provides a system Uta. allows the sterns administrator to .dent, y, andm 
Llistic views, ,0 see relevant parts of the network, and to see its status and configurat,on. Thrs 
faciUtatesdiag^osisandcorrectionofanyproblemeffeoavelyidentifiedbyuseofd-en^^^^^^ 

tools and by directly activahng m^pulation and control software to cor^t the problem or 

adiust the operation of the object. 

nisanotherobjectofthepresent — .oachieveadministrationsystemswhtchhave 

other valuable feahues: an integrated operabiUty that enables each .imction to work seaml^sW 
with the others; a common mode, for administering all aspects of systems managemen w,th h 
same look-and-fee, tor all functions; an open and interoperable solution that works across 
platfom,s, complements network managers, and easily comrects to other soluuons; a robus 
pL s. of systems management functions ^ meet all the basic needs for mana^ 
client/server systems; and a customizable interface that can be tailored to meet the present and 

futureuniqueneedsofdifferentuserswithinacompanyororganrzanon. 

A further objective of the present invenUon is to broaden the scope of the systems under 

management, providing a comprehensive and business-orientcd view of a Ml en.erpn. 

neJrk. The invention describes in virtualreality terms thehierarchicaistrucmreoan^^ 

The present .nvention includes a hierarchical or^zation of the various ™" 
.ystl components, including continents, wide area networks, citie. butldmgs. su^n^^^ 
inents, computers and peripherals, and their interna, hardware, firmware, and s^ftw^ 
rlrces. However, another objective of the present invention is to provide a sys^m .a. do^ 
no. impose on the user any particular hiera^hical model. The present mvenfon allows the use 
of configuration tools em*ling the user to set up any logical structure. 
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C -.i-.. vrnc«. Vtem. Business process views filter toe realistic perspectives 
necessary to reflect on a specific busirtess ta.«est, aUowing a manager to review and manage a 
world that contains only computer-related assets relevant to that interest (payroll, inventory, cost 
accounting, etc.). Close integration with the monitoring and administration Mities gtve 
immediate access to serve, and workstations, reflecting their curent sta«rs and providing fine- 

grained remote control. 

In (he present invention. Business Process Views allow users to customize the mvenhve 
system to dynamicaUy construct filters to view re^urces as they pertain to unique busmess roles 
or functions, business applications, locations or geo^phies, or any traditional resource v.ew^ 
This concept inverts the traditional resource-centric view of ent«prise management mto a logical 
view, mapping managed resources needed to a specific business perspective. For example, 
views include but are not limited to. one or more of the following: geo^aphy or location such as 
Northeast U.S. applications; a Actional role such as that of an administrator or secunty 
manager; any business appHcation such as an inventory or payroll; any community of ,nt«est, 
such as all users interested in a specific set of resou^es; and resource views such as a database, 
network, or a server, or any combination of the above. 

This allows the user to identify the parts of the network that relate to a specific bustness 
interest such as inventory control or payroll, and to display those parts in 3-D virtual realtty 
enabhng ttte user quickly and intuitively to identify and solve a problem with a payroll server. 

D AnnHcabilitv. The present invention can be applied to the management of 

any syst^n consisting of devices capable of some form of industry standard network 
communication, including dial-up networking. Such devices include but are no. limtted to: 
manufacmring. refining, and chemical processing equipment; air conditioningmeatmg systems; 
automated pdson door and other secuhty systems; electidcal Hghting systems; forkltft systems; 

travel systems; and elevator systems. , ^ ., j 

The present invention will become more Mly disclosed and understood from ttie detafled 
description given herein, and ftom the accompanying fi^es. That description and .hose figure 
are provided by way of illusttation only. Changes. modifica.ions. implementations, and 
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embodiments obvious to one skilled in the art given the within disclosures, are within the scope 

and spirit of the present invention. 

6. Brief Descriptioii of the Drawings. 

Figure 1 is a global diagram showing the relationships between the various system 
components used in conjunction with the present invention. 

Figure 2 is a flow diagram showing the operation of the main control loop of the virtual 

reaUty workstation software system. 

Figure 3 is a flow diagram that describes the processing of and the responding to various 

events. 

Figure 3A is a flow diagram that describes the processing of status change event data 
shown in Figure 3. 

Figure 4 is a flow diagram the presents the algorithm used to determine the next position 

of the virtual reaUty system view. 

Figure 5 is a flow diagram that describes the process used to adjust each system model 

according to the viewing position. 

Figure 6 is a flow diagram that describes the rendering of each visual object. 
Figure 7 illustrates the visualization workstation Control Panel. 
Figure 8 illustrates the Business View control panel. 
Figure 9 illustrates the manual navigation control panel. 

Figure 9A is a diagram showing operational features of automatic navigation used in 

connection with manual operation. 

Figure 10 presents an overview of the operation of model management tools used to 
configure the visual appearance of various system components displayed in the virtual reality 

system. 

Figure lOA illustrates the Class Editing and Definition panel of the system presented in 
Figure 10. 

Figure lOB illustrates the Properties Panel of the system presented in Figure 10. 
Figure IOC illustrates the SysObjID Panel of the system presented in Figure 10. 
Figure lOD illustrates the Menu Panel of the system presented in Figure 10. 
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Figure lOE illustrates the Cursor Panel of the system presented in Figure 10. 
Figure lOF illustrates the 2D Icon Panel of the system presented in Figure 10. 
Figure lOG illustrates the 3D Icon Panel of the system presented in Figure 10. 
Figure lOH illustrates the Selecting New Object Panel of the system presented in Figure 

10. 

Figure 101 illustrates the Selecting File Panel of the system presented in Figure 10. 

Figure lOJ illustrates the Colors Panel of the system presented in Figure 10. 

Figure lOK illustrates the Textures Panel of the system presented in Figure 10. 

Figure lOL illustrates the Size Panel of the system presented in Figure 10. 

Figure lOM illustrates the Distances Panel of the system presented in Figure 10. 

Figure 11 illustrates the system with Status Display and a network scene. 

Figure 12 illustrates the Targeting Reticule. 

Figure 13 is an example of a World View depiction. 

Figure 14 is an example of a map scene depiction. 

Figure 15 is another example of a map scene depiction. 

Figure 16 is an example of a building scene depiction. 

Figure 17 is an example of a network scene with bridges and routers. 

Figure 18 is a depiction of component interior scenes. 

Figure 19 is a depiction of software processes and other software subsystems in a 
computer. 

Figure 20 is a screen display illustrating the zooming graph features of the present 
invention. 

Figure 21 is a flowchart illustrating the steps of one embodiment of the method. 
7. Detailed Description of the P .-Pf*>rrpri Embodiment 
A. ronstructinff the Invent ive System 

The invention is accomplished by use of the 3-D graphical user interface, network 
discovery and monitoring software engines that interact with and enable the interface and a 
central repository, and a central repository comprising a comprehensive database describing 
every computer-related asset on a network. 
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In fte preferred embodiment all of (he objectives of a.e present invention are 

accomplished. 

Architecture 

Tl>e various components that comprise the complete network analysis system ts shown m 
Figure 1, and includes one or more of visualization workstation 101. an object repository 102 
one or more management applications 103, and one or more agents 104 on each such 
management appUcation. Tlte visuaHzation workstation interacu primarily w,th the object 
repository 102: it requests information ftom it. it sends commands to it, and it gets not,f,cat,ons 
of events such as status changes or object additions from it. Tlte repository 102 in mm gets tbts 
infonnation from the various management subsystems 103 which are fed by the agents 104 or. 
the managed systems. The key architecu^ral consideration of the present system is that tn no^d 
operation, tite visuahzation workstation 101 interacts only wiflt the object reposttory 102. Tht 
^nimizes network trafHc, optimizes the perfonnance of the rendering on the workstation, and 
minimizes the interconnectivity between the visualization workstation 101 and the multin,de of 
management subsystems and agents existing in practical networks. 

On rare occasions, the visualization system sends commands directly to management 
systems and gets even, notifications directly ftom management systems (or indeed ftom any 
other application on me network). Uus, tite architecture is designed for optimal operation and 
minima, network load in nonna. ope^tion. wititout imposmg limitations on tire forms of 
communication possible in special cases. 

The main program operation and display management process is show by Ftgure 2. TTie 
program operates in a loop, repeatrfly performing the same functions until the user termtnates 
fl,e program. Tlte loop begins by receiving and respondmg to events shown m module 201. If 
ti,e even, received is an Exit command, the loop terminates. Otherwise, the loop continues by 
determimng a new position of observation 202. Next, the visible models are adjured U, reflect 
any changes in position 203. Finally the graphical objects are rende^ 204. In order to acheve 
smooth animation, h is important that this main program loop executes as quickly ^ poss,ble. 
The ideal rate of execution is 30 repetitions per se^nd, which corresponds to a video frame rate. 
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Figure 3 elaborates on module 201 of Figure 2. This module deals with the system 
responding to events. The five modules shown at the top of Figure 3 represent the different types 
of events the system receives. These include user interface events 301, messages from other 
parts of the virtual reality workstation 302, messages from third party extensions installed in the 
virtual reality workstation 303, event notifications received from the object repository 304, and 
messages received from other systems 305. All of these events and messages are processed by 
the event dispatcher 306, which calls appropriate code modules to act upon the events and 
messages. These include a module to stop the current flight 307, a module to begin a new flight 
308, a module to change the visualization 309, a module to handle a change of status 3 10, and a 
module to perform specific operations on objects 311. 

Figure 3a elaborates on status change 310 of Figure 3. The status change event 320 
message is sent to the event dispatcher 321 which communicates with the module 322. Said 
module 322 sets the appropriate model of the appropriate color for the status indication of the 
affected object. A decision is made in module 323 as to whether a preset threshold for 
visualization has been exceeded with either the status indicator being hidden at module 324 or 
the appropriate change of status signal being sent. Figure 4 elaborates on module 202 of Figure 
2. This module deals with the system changing the position of the user's observation. This 
module begins by determining if an instant jump must take place 401. If not, the system 
determines if the viewer should enter or exit a scene 402. If not, the system detemiines if 
automatic flight mode is active 403. If automatic flight mode is not active, the system calculates 
the next position and orientation based upon the input control devices and the rate of frame 
rendering 406. If automatic flight mode is active, the system calculates an interpolated position 
and orientation along a calculated flight path 407. If module 401 determines that an instant jump 
must take place, a determination is made if the jump is to a different scene 404. If so, or if 
module 402 determined that an object must enter or exit a scene, the system determines a list of 
visible objects in the current scene 405. Finally, the system detennines a new position and 
orientation 408. 

Figure 5 elaborates on module 203 of Figure 2. This module deals with the system 
adjusting models to reflect any changes in position. Modules 501 and 506 handle the iteration 
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through the Ust of visible objects, selecting each object to be rendered. Module 502 determines 
if the object is opened in place. If it is, module 51 1 determines if the object should be closed 
and, if so, modules 512 and 513 delete any contained objects from the list of visible objects and 
replace the closed objects with the appropriate model. If module 502 determines that the object 
is not opened in place, module 503 determines if the object should be opened in place and, if so, 
modules 509 and 510 replace and add needed objects. If module 503 determines that the object 
should not be opened in place, module 504 determines if the object should be adjusted for level 
of display and, if so, invokes module 507 to replace the object's model. Module 505 then 
detennines if the object should be resized and, if so, calls upon module 508 to resize the object 
model. Finally, module 506 retrieves the next visible object, iterating through the entire list. 

Figure 6 elaborates on module 204 of Figure 2. This module 601 performs the actual 
graphics rendering of all visible objects. Objects are rendered in the invention using a graphics 
accelerator. When available, and in other embodiments, however, sufficiently fast main frame 
system processor(s) could be used to perform the rendering. The present invention performs the 
rendering using the OpenGL graphical interface library. This library is structured such that the 
caUing program need not be aware of the underlying graphical hardware. The use of a software 
implementation of OpenGL on current microprocessor-based systems, however, will result in a 
speed penalty. 

The object repository 102 in Figure 1 is notified of major changes in the system 
configuration or status (changes to those objects it maintains) through the standard event 
notification mechanism of the invention. Because only major, relatively static objects are 
maintained in the repository, the real worid interface is kept up-to-date on important changes 
while network frafSc is limited. 

When the workstations require dynamic data, which is maintained only on the remote 
SMS databases and not replicated in the repository, the repository server passes their requests on 
to the remote systems. The workstations can access all data, whether stored locally or not. This 
allows the system to balance the conflicting requirements. For example, important servers may 
install monitoring agents to report continually the status of a database server. This information is 
already monitored centrally, with event notification over the network, and displaying these 
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monitored processes centrally does not burden the network excessively. But if the user asks for 
visualization of all the processes running on the server, the system makes an on-line query to the 
machine; this query, which does burden the network, occurs only when requested. 
Technical Specifications 
5 In the standard configuration, there will be one object repository server shared by several 

workstations; the object repository communicates with the distributed management facilities 
across the heterogeneous network, and the workstations communicate only through the 

repository server. 

VR Workstations 

LO The operating system is Windows NT. A Unix system may be supported in other 

embodiments. The preferred hardware embodiment includes a personal computer utilizing not 
less than a Pentium 586 microprocessor by Intel. The computer should contain at least 32 Mb of 
Random Access Memory and a 3-D accelerated video board with OpenGL support. The 
preferred system should include a powerful workstation rumiing the Windows NT operating 

15 system. The preferred embodiment uses the standard Open GL 3-D rendering facilities provided 
in Windows NT; for good performance, the platfonn should provide hardware acceleration of 
OpenGL, which is provided by a number of vendors including Intergraph. 
Object Repository Server 

The Operating System is Windows NT. A UNDC system may be supported in other 
2 0 embodiments. 

The hardware is an Intel-based PC. Other hardware platforms may be supported by other 
embodiments. 

It is possible to execute both visualization and repository on the same system; in that 
case, a dual-processor system is preferred. One or more visualization workstations can also work 
25 with an object repository operating on a separate server machine. 
Network Connection 

The system supports several network connection protocols to all systems that will 
generate events or feed data into the repository, including TCP/IP. SNA and DECnet. The 
repository server uses TCP/IP to communicate with the VR workstations. 
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Database 

The current embodhnent uses Microsoft SQL Server. Other industry-standard databases 
may be used in other embodiments. 
S ystems Con figuration 

The standard configuration combines a single object repository server with one or several 
Real World Interface workstations. Removing the database processing and event handlmg from 
the 3-D simulation reduces its impact on the performance and realism of the simulafon. The 
object repository server can operate on the same machine as ote CA-Unicenter processmg. A 
minimal configuration might combine the object repository server and a Real World hrterface 
workstation on a single machine, a. some possible impact on the performance (and hence 

reahsm) of the 3-D visuaUzation. 

The inventions described above may be varied or implemented in many ways. Vartattons 
and implementations as would be obvious to one skilled in the art are within the scope of such 

invention. 

In other embodiments of the present invention, advanced display options are provtded, 
including an imme^ive display with head-mounted displays, and a cave display with mnlfple 
large screen displays encompassing the user. 

B. Elements an d Features 

Real World Interface 

The syst^n and apparatus of the present invention displays an entire network of 
computers, peripheral equipment, operating systems and application proyams in an envrromnen. 
that represents physical reality: the geographical space in which the network exists, which maght 
span several continents and countries and might contain various regions and cities and groupings 
of buildings (often called "campuses"), a particular building, a particular floor of a burldmg. and 
a particular room and the computer related units in the room. In order to achieve the appearances 
U,at are important features of the present inv^rtion. in addition to the exterior of the compute, 
the inside of the computer with intental components such as the processor, the drsk storage, 
network card, tape sU.rage, etc.. are displayed in virma. reality. In addition to the computer 
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devices the networks in the present invention processes, databases and other abstract objects are 

rendered on the display as real things. 

The realism of the inventive system is expanded by the use of photo-realistic buildings 
with management tools so that the user may be able to feed photographs of the user's buildings or 
floor layouts and equipment into his system. The inventive system includes support for three 
dimensional models produced by industry standard three dimensional modeling tools. The 
inventive system also provides simple modeling tools to create new simple models. 
Management tools to identify computer-related units by class or category, such as a Hewlett 
Packard printer or an IBM server, are provided. 

The present invention provides to the user a control panel as illustrated in Figure 7. 

Tarfyetin p Reticule 

To identify individual objects, the Real World Interface uses the idea of an intelUgent 
cursor or "targeting reticule" that displays information about the indicated object, as shown in 
Figure 12. Illustrative infonnation includes the network address and the name of the system. 
Cities buildings, subnetworks and computers are not labeled in the 3-D view, because 3-D text is 
hard to read. Instead, the mouse cursor becomes a "targeting reticule" which displays 
infonnation about the object the user points to. It displays the information "Hudded" (a new 
verb, coined from "Heads-Up Display") onto the "cockpit window" or crosshairs/quadrant 
display. 

By simply pointing to an object with the pointing device (such as a mouse), the user can 
bring up a reticule that gives the formal and informal name for the object and a brief summary of 
its status. THis technique works for all objects, from cities and buildings, to networks and 
computers, to disk drives and processes. 

ttpalism Enhancers 

The inventive system uses other features to enhance the illusion of reality, including the 
provision of geographic maps to provide backgrounds, such as realistic 3-dimensional 
topographical surfaces, which, through texture rendering, creates more useful views and user- 
specifiable maps or textures for arbitrary geographic regions that allows a customer to define a 
geographic area of interest. 
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Antnniatic DetectioP of ToDologv romponents 

The configuration of the current invention requires the automatic detection of network 
topology and devices, and utilizes the automatic detection of internal computer components and 
of software processes. Further, the current invention includes interactive management tools for 
configuration of geographic relationships, buildings and network relationships. The present 
invention allows the override or the custom tailoring of the computer system and the network 
topology when automatic discovery fails, or produces unsatisfactory or incomplete results. The 
current invention also includes an automatic layout of logical networks and 3-dimensional space 
and an interactive layout of network and devices over floor plans or other diagrams. 

rnmmon Internal Structure 

hi the preferred embodiment of the present invention, a common intemal stiructiire is 
provided to allow botii 3-dimensional enviromnental, 2-dimensional and standard user interface 
displays like tree diagrams, icons and folders. This is critical to allow a user to operate tiie 
system even when sufficient computer power is not available for a 3-dimensional display, or 
when other reasons dictate tiie use of other interfaces. 

rustomizability 

An automatic layout and 3-dimensional realism is provided to lay out logical networks, in 
3-dimensional without criss-cross lines. Manual configuration capability is also provided. Figure 
10 presents an overview of the configuration process. The present invention provides to the user 
a series of panels to achieve customization. 

The Class Editing and Definition user interface illustrated in Figure lOA allows the user 
to select a class to work with, or to create a new class of object to be used in the system. 

The Properties tab in the user interface iUustirated in Figure lOB allows the defmition or 
modification of properties of tiie class, and assigmnent of values to tiiose properties. 

The SysObjID tab illustrated in Figure IOC provides for specification of ID numbers to 
be used in communication with tiie system's own programs and witii program extensions built by 
third parties. 

The Menu tab illustrated in Figure lOD provides for defining the menu that is displayed 
when activating an object of tiiis class, and tiie actions to be taken for tiiose menu items. The 
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actions can include conmiunicating with built-in faciUties of the system, and executing other 



programs. 



The Cursor tab illustrated in Figure lOE provides for specifying what data should be 
displayed in the four quadrants of the cursor, the targeting "reticule." 

The 2D Icon tab illustrated in Figure lOF provides for specifying the icon to be displayed 
in the 2-D interfaces of the system, for different status values of the object. 

Tlie 3D Icon tab illustrated in Figure lOG provides for specifying the 3-D model for the 
object, to be used in the 3-D visualization system. The model currently selected may be 
previewed in the window on the left at Figure lOG. The control panel on the bottom of Figure 
lOG allows for adjusting the view or the orientation of the object. The system also allows the 
user to select each of the various models used in the adaptive display ("level-Of-Detail" and 
"Open-in-place"). 

The Selecting New Object view illustrated in Figure lOH allows the user to create a new 
object from simple geometric shapes. This model may then be adjusted in size, shape and 
orientation, and decorated with colors and texture coverings. 

The Selecting File view illustrated in Figure 101 allows the user to select an existing 
model generated with an industry-standard modeling tool. 

The Colors view illustrated in Figure lOJ allows the specification of the color of the 
entire object. The Textures view illustrated in Figure lOK allows the user to specify the texture 
map (bitmap) to be pasted onto the object to give it a photorealistic appearance. The textures are 
bitmaps in industry-standard formats, and are often scamied photographs (although drawn or 
painted images may also be used). 
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The Size view illustrated in Figure lOL allows the user to adjust the size and shape of the 

object. 

The Distances view illustrated in Figure lOM allows the user to specify the distances at 
which the different models are switched in, under the Level-of-Detail and Open-in-place modes 
of adaptive display. The interactive layout of network and devices over floor plans or other 
diagrams allows a customizing function by which the automatic layouts of logical networks can 
be shown in relationship to floor plans or other diagrams. 

n Ynamic Rescaline 

Dynamic appearance, navigation and behavior during execution are provided by the 
current invention. Network comiections are shown and various parts of the network are 
automatically rescaled as the operator moves through the realistic. 3-dimensional enviromnent to 
get closer to the part of the computer-related units which are of interest. Network comiections 
and indicator lights are initially shown large enough to be visible in the overview, but as a user 
travels in virtual reality, closer to a particular object, they unobtrusively shrink to take on a more 
reasonable size in the local view. This automatic rescaling does not continuously scale a 
network comiection down to the actual size of a cable. The external view of the geographic 
space is tiie most severe scaling problem. 
Automatic Navigation 

hi the present invention, navigation occurs automatically by selection of a device in a 3- 
dimensional enviromnent, in order to retain the illusion of residing in real enviromnent. An 
automatic navigation control panel is provided as illustrated in Figure 9. Th. system provides a 
»you are here" display, indicating the present location in terms of level of depth in the hierarchy 
and indicating the choices made to reach the displayed level. The navigation portion of the 
inventive system allows the user to select and to navigate to higher levels within the hierarchy. 
This automatic navigation includes automatic detemiination of a reasonable trajectory, avoidmg 
collision with intervening objects such as buildings, and automatic detemiination of a reasonable 
speed and reasonable acceleration and deceleration that will take a separate amount of time for 
the user. The invention also provides for a history log and search windows using the user 
interface techniques well known in the computer industry. A history log will enable the user to 
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view recently visited locations and quickly jump to a desired location. Search windows allow 
the user to search the network for the location of a particular unit, based on name, address, node 
ID or other properties (using well-known database search techniques). 

When a GUI screen shows some important data, such as the event log which lists critical 
alerts a "take me there" button automatically flies to the computer that originated the event. 

' The mouse provides "automatic flight" in a logical extension of the classical mouse 
operations. Moving the mouse over an object (without clicking) displays information about 
just like the prompts displayed by modem toolbars and other controls: this is the "targetmg 
reticule." Clicking on an object means "take me there:" it m^es the system travel to the object 
through a smooth flight path and halt in front of it (no disconcerting jump). Double-chckmg on 
the object means "enter the object," as does a second click after the first travel. Right mouse- 
click brings up a local menu, common in modem GUI systems. 
Manual Navigation 

For manual navigation in 3-space, preferred embodiment of the invention calls for a 
WK-typ. 6 DOF (degrees of fteedom) control device, such as the Spaceball, that allows 
independent control of both position and viewing direction. Both allow control of movement m 
3 dimensions (forwardAack, left/right, up/down) as well as nmiing the direction of vew (pitch, 

yaw, roll). 

Manual fUght, may be accompUshed by use of a standard mouse with push buttons. Hie 
systems provides a control panel for manual flight under mouse control. While certainly less 
flexible than the 6-DOF devices, the control panel illustrated in Figure 9 is quite useful 
especially in combination with automatic flight. 

Certain feattires of automatic navigation maybe used after use of and in comiection with 
manual navigation, and these features are illustrated in Figure 9A. IHese features allow the user 
to navigate manually down into the hierarchy at a specific geographic location, to jump by a 
"take me there" request, by a search or by use of a tree structure, to a second geographic 
location. The user by manual navigation can ascend the hierarchy in either location with the 
"you are here" feature of the manual operation, 
rnnfiniial Reporting 
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Continual reporting is provided by the present invention, including a status display of 
devices. The continual reporting function of the present invention is further achieved by the use 
of distributed originating-site filtering and the reduction of status display in the network. 

Tntelli pent Aggregated Statu s Display 

The present invention provides a system that indicates the status of objects by use of 
colored indicator lights. The status reflects what is going on inside computers, operating 
systems, networks, disk drives, databases and critical processes. Such status indicators are 
aggregated so that network segments, subnetworks, buildings and cities reflect the status of what 
is in them. At the highest level, when traveling over the map, status indicators show the 
aggregate status for cities and buildings, in the form of globes that hover over the objects. This 
is shown in Figure 1 1 . 

Only problems are indicated: to keep the scene simple, green Ughts indicating OK status 
are omitted. The aggregation is intelligent, weighing alerts based on importance, to avoid 
everything always showing red, a problem with early network management systems. The 
invention discloses that the view inside a building reflects the aggregate status of subnetworks, 
segments, and eventually the individual machines. Again, they are shown with hovering colored 
globular Ughts, and show only problem spots. Inside a computer, the systems show the status of 
components and subsystems. Our indicator shows the status of the computer itself, in terms of 
loading, process queue length, and number of users, while the status of its subsystems are 
indicated separately on each one. 
Aria prive Disclosure 

The inventive system utilizes several techniques to adapt the level of detail in the view to 
particular circumstances. This is necessary because of the performance and resolution 
limitations of today's hardware, and to make the display comprehensible to the user. Today's 
computer systems cannot visualize the thousands of computers in a country-wide network with 
adequate speed; even if it could, it would do little good because from 30,000 feet a computer is 
no larger than a pixel on the screen; and even if it were visible, the user would not want to deal 
with a large scene with thousands of objects in it. 



{M-A1079.DOC;1} 



23 



The system uses three techniques to deal with this problem. First, the 3-D visualization 
uses the standard technique of "level of detail," where several models of different complexity are 
provided for each object. A distant object is rendered with the simplest model; as the user 
navigates closer, the system automatically substitutes increasingly complex and realistic models 
as resolution warrants. Second, certain aggregate objects such as a network segment 
automatically "open-in-place» to show their contents as the user gets closer, and are replaced 
with their closed external model again when the user moves away. Third, some complex objects 
remain closed and must be entered to show their internal components. 

To avoid irritating flicker, the switching for "level-of-detail" and «open-in-place" are 
implemented with hysteresis, where the switching out distance is greater than the switching m 
distance. 

The inventive system is fully configurable in that the user can specify which class of 
object can open in place or provide several models for "level of detail" display. 

The user of the system can ignore this issue - when it is done well, it is unobtrusive, 
simply speeding things up - but it gives a system administrator an opportunity to tailor the 
presentation to the users' interest, to the system configuration and to the performance of available 
hardware. 

gf^t..QMniiitormg.F "^^^i"r ^"'^ Aggregation 

The present invention communicates with prior art technologies which continually 
monitor the operating status of all the components in the system: hardware and software, 
network and operating systems, databases and applications, network cards and disk drives. The 
results of the monitoring are then filtered according to preset threshold parameters and 
aggregated per the user's specifications. 

Monitoring AgeD ^^/^p*-" Architecture 

The subsystems are monitored by independent agents on the managed systems; the agents 
report back to a manager whenever there is a significant status change, and possibly on a regular 
basis to signify that all is well. The invention provides customizable agents, but it also supports 
industry-standard protocols such as SNMP, allowing third-party software agents and hardware 
devices to be managed. 
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Filtering of Sec nndarv Problems 

Intelligent filtering allows the system to remove the noise, eliminating secondary problem 
reports when a fundamental problem has already been detected. 
Aggregation 

Although the agents monitor all the individual components, the system reports aggregate 
status for larger systems: for an entire computer, an entire network, an entire building, an entire 
country. ITie aggregation permits weighing factors, reflecting the reality that a database server 
machine is more important than an individual desktop machine. 

Alternative DiSPlaVS 

The status of all components, from large aggregates like cities, buildings and networks to 
individual components like routers, computers, disk drives and databases, is displayed with the 
same principles in the virtual reality view, in the diagram view, or in the tree view. 

Business Process Views 

The present invention also visualizes information technology assets from a specific 
business perspective. The invention enables an isolated view of service levels, problems and 
administration for specific interests such as order entry and payroll. These business-onented 
views of the assets in the network are based in groups. TTiese are arbitrary groupings of thmgs, 
groupings that make a specific business viewpoint. The user defines these groupings usmg 
simple drag-and-drop operations in the configuration subsystem, using standard GUI technology, 
nie invention fiirther permits the definition of any arbitrary grouping of computers, segments, 
subnetworks, routers, databases, and applications which may be assigned to a folder. 

Rnsiness Process Filtering 

The system provides a separate control panel, illustrated in Figure 8, that shows the 
aggregate service views (the user configures this panel, selecting the service view important and 
should be continually monitored). The services views have backUt buttons. The color of the 
backlit button represents the status of each business view. 

The selected view becomes a filter for the system, one that addresses only those objects 
that exist within the selected service view; others sunply disappear from view. This applies to all 
levels of hierarchy: if a city has no components related to that service view, or if a subsystem, a 
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segment or a computer is not involved with the subsystem, they are not part of the business view; 
similarly, if a process or database is not used in an application inside the drive bay, it is removed 
and is not part of the business view. 

nirftftiv Visuali/inP Busines s Groups 

It is also possible to group several computers, segments or subnetworks in a group and 
place this group in a building, at a subnetwork or segment to give the manager a perspective of 
the resources in the system that represents the physical comiectivity of the network: it does not 
show which computers are comiected to each other, but it groups computers or networks 
according to organization or project. 

Control Panel 

An information display control panel is illustrated in Figure 7. Hie information display 
panel is configurable, like the other control panels; it may be turned on or off, and placed where 

it is convenient. 

Dis play of Obje ct Properties 

The Real World Interface provides built-in search faciUties that use an ordinary GUI 
screen and provide immediate auto-flight, highlighting and filtering of specific objects. The 
Real World Interface also automatically invokes the standard interface facilities for manipulation 
and control of the machine under focus or other objects (user ID's. installed software, files and 

backup media, etc.) 

Multiple Views 

The Real World Interface provides two additional views of the resources in the networks 
and the business groups: a two dimensional map or system diagram representing the system as 
comiected icons, and a tree diagram representing the hierarchical structure of the network. 
These views are usefiil as navigation and search aids firom the 3-D view. T^ey are also robust 
enough to work as the main interface when using a low-end computer not capable of showing the 
3-D view-for example, when logging into the system fi-om home 

Mani pulation and rnntrol of the Managed System 

The Real World Interface invokes the standard GUI facilities for manipulation and 
. control of the managed objects. Though a local menu, the user can bring up manipulation and 
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control panels for each defect. From this panel, the manager can reach every management 
facility available for the targeted machine. 

FYfPndahililtv bv APT System (Qp en Architecture) 

The inventive system provides an API system that allows the user to extend the interface 
and object capabilities of any part of the inventive system. The API system allows a new object 
to be added or a new class of objects to be defined in the object repository, information displayed 
in the targeting reticule to be modified, the user interface displays to be modified by 
conventional manipulation tools, or the colors for status indication to be changed. Menu options 
for the new object or class of objects can also be controlled. 

Performance And Loading 

In other embodiments, the present invention may provide a system that illustrates the 
amount of activity on disk drives, network cards, etc. by use of a blinking hght, similar to the 
drive light on a real computer. The local agents then monitor the activity on the system, and 
report average loading. The system may be configured for different levels of timeliness, a 
typical setup might report statistics on a twenty-minute basis. Thus, the activity indicator shows 
what is happening with the system on an average basis. 

C. Dynamic Operation of the System 

The system of the present invention starts with a view of a typical system administrator's 
area of responsibiUty as a system manager - the entire earth - rotating before him or her. Next, 

the system opens up a world map. 

From there, the user may navigate closer to an area of interest, either by flying with 
manual control, or with auto pilot: if the user clicks on the map the system will fly the user to 
the selected location. 

As the administrator gets closer, he or she sees a relief map with cities and network 
comiections. Again, the administrator can fly manually, using skills as if a heUcopter pilot, or 
click on a city to get flown there by auto pilot. 

Normally, all the cities, buildings and networks in the network are shown. To reduce the 
complexity, the administrator can activate a business view which shows only what is relevant to 
the specific business interest or problem of interest at any particular moment. 
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Eventually, as the administrator gets closer to a city, he or she sees buildings. Each city 
and building reflects the aggregate status of the systems inside it. in real time, by the status Ughts 
hovering over them. As the administrator flies into a building (or doubleclicks on it) he or she 
sees e.g., the LAN configuration inside the building or other network scene. This network scene 
shows thl actual computers, printers, routers and bridges comiected to the network: as soon as a 
new computer is comiected to the network, it becomes visible to support discovery services and 
appears in this view immediately or after a regular refresh, depending on how the system is 
configured. The system reflects the entire network hierarchy, showing internetworks, 
subnetworks and segments. TTie user can fly around among the computers, identifying all 
resources and observing their status. The system shows computers, routers, printers and other 
devices as realistic models. The status of computers, components and software systems on a 

continual basis is available data. 

If the administrator flies inside a computer (or double-clicks on it) he or she sees a view 
of the inside of it, with the relevant subsystems: a tape drive, the disk subsystem, the processor, 
the network card, and the aggregate of software processes and other software subsystems. 

Entering a subsystem shows a view of what is going on inside it. For example, the 
software space contains processes; the system shows all of the monitored processes, displaying 
their real-time status, size, resource consumption, etc. The management system continually 
knows the state of the monitored processes (database management systems and other important 
servers) through the operation of agents on the target machine. 

Similarly, the disk subsystem shows all the logical drives ("file systems" in UNIX 
terminology) known to tiie system, whether local or attached from a server. It shows their status, 
size and free space (shown through the targeting reticule). For remote drives, the admimstrator 
can easily navigate to the system that owns the drive. For local drives on a server that are 
attached from other machines, the adminisfrator can easily get a Ust of the cUent machmes and 
navigate to them. 

Once in a computer, the user can enter each subsystem and inspect its properties and 
status in real time. Clicking on a subsystem such as a disk drive or a database brings up the 
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sundard GUI management adUfe. giving the administrator direct aocess to both operational 

and administrative aspects. 
Ma p Scenes 

The world map (as illustrated m Figure 13) allows the administrator to check the area of 

"""''' a map of each region (as illustrated in Figures 14 and 15) or continent shows the major 
cities and network links. Ue us« controls how the network is displayed at this level, using the 
configuration tools: the user may want enough detail to be useM. but not so much that he or she 

drowns in network links. 

Each "city" really represents a local region, which may contain several towns and cittes. 
For example, the system may be configured so that "New York" includes New York Oty as well 
as Fort Lee and Newark in New Jersey, and "Boston" includes some of the Boston suburbs. 

AS the user gets closer to a certain region, a regional map with hi^er resolution and 
„ore detail is automatically inserted (an example of "Level-Of-Detai," display). These maps 
nray be tailored to the usefs particular interests, showing specific towns, highways or nvers as 
the user may prefer, by using the configuration subsystem. 
Riiilrting Scenes 

city symbol is opened up to show the buildings (illustrated in F.g^ 16) when fte 
^ gets Close, while other cities remain as simplified objects. If two cities are close together 
(such as 1.S Angeles and San Diego), both may open up into buildings. TTre butldtngs are 
located at r^nable, user controUable positions, but the scale is not realisUc; at a realrstte scale, 

the buildings would be too small to see. 

TT,e system contains a number of standard buildmg desi^, but the user can enter custom 
designs using the configuration utility. TWs means a user can take photographs of Us o,™ 
buildings, feed them in as bitmaps together widr a g«metry design (basic dimenstons), and make 
its buildings look like the real thing. 
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Nfttwork Scenes 

The system reflects the network hierarchy: the initial scene inside a building shows the 
various subnetworks and routers, when the user enters a subnetwork, he or she sees the various 
segments and bridges, and eventually sees the computers and other devices attached to the 
opened segments, as shown in Figure 17. This is done for practical reasons: a horde of 2.000 
computers is not manageable, nor can the computer render them effectively. V.. hierarchical 
network structure gives the user a way to select only the necessary information. 

The subnetworks are comiected by routers, and the segments by bridges - all of these are 
manageable devices, and their identity and status are shown. 

The segments open up in place as the user get close to them, showing all the computers, 
printers and other devices. The visualization illustrates the structure of the network: a ring like 
Token Ring or FDDI, or a bus like an Ethernet. 

The rendering is optimized by simplifying the computers that are far away, and 
automatically restoring the more precise representation as you get closer (another example of 

"Level-Of-Detail" display). 

Tl,e system automatically generates a reasonable layout of the network and the 
computers. The user can also define the layout manually, using the 2-D layout and configuration 
utility The user can provide a picture, for example a diagram of an office layout or a simplified 
campus map, for use as the floor instead of our standard tiles; this can help in using the system 
by associating subnetworks and computers with their physical location. 

Device Scenes 

The system knows how the different devices loolc: PCs, UNIX workstations, servers, 
mainframes, printers, routers, etc. The visuaUzations of the devices are very realistic, based on 
texture mapping (photographs pasted onto the S-D models). The models are complete, even the 

backs of the devices look correct. 

The database of physical models is maintained to reflect the common devices. As with 
buildings, the user can add new computer types by taking photographs of the machines (all the 
sides, including the back), scan the images, clean and simphfy them, and define a new computer 
model with a geometry defmition and these images. 
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rom puter iF itgrior Scenes 

Mos, of flie component mside fte computer are active: the CPU, the network card, the 
drive bay and the software space. All may be displayed in virtual reality view, as illustrated m 

Figure 18. 

Additionally, g«phical displays of software process and ofl>er computer processmg 
activities are provided, as shown in Figure 19. 
n. y^oming f ^raph Diagram 

The system combines the capabilities of two types of user interfaces, graph diagrams and 
continuous zooming, in a unique way. At the highest level, ti>e elemenu of a system ts 
represented as a graph dia^am, with icons intercomtected with lines. user can seamlessly 
zoom into the diagram, and pan the dia^ in any direction to make visible any part of tite very 
large virtial space. As the user zooms in to Ute dia^am. and the icons g« larger, the .cons are 
automatically r^la^ with their internal structure. Fi^re 20 provides a illustrative screen 
display employing the graphical zooming and display techniques of the system. Refemng to 
Figure 21, there is illustrated the steps of one embodiment of the described method. 

Since 4e user interface represents a graph of mtercomtected objects, and no, just a set of 
objects arranged on a desktop, fte diagram may a. ^y moment contain Imes that connect the 



icons. 



The imier connections in tite contained graph sdircture appear as the user zooms in, and 
disappear as flie user zooms out, just as the imier nodes do. 

AS is common in modem user interfaces, both the type and properties of the objects or 
interconnections may be represented visuaUy, using ^hical elements, coloring, amtotafon or 

animation. , . 

Different types of graph structi«es are exti:emely common in computer systems and m 
any other fleld of human endeavor. The techniques employed by appUcants' system apply to any 
data sttiicture that may be represented as a graph. 

In one implementation of the dynamic high-speed zooming featine, data for certam 
display elements, such as icons, may be stored in a database for association with specific data to 
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be visually represented. In such an embodiment, the display element data may be retrieved from 
a local system or database or from a remote system or database, such as a remote server. 

In such an embodiment, it is preferable if the data retrieval and graphic zooming 
operations may be executed asynchronously. In cases where the display data is refrieved from a 
remote system, this operational autonomy enables a workstation to seemlessly execute the 
zooming operation even if the data retrieval process is slow. For example, if a workstation has 
requested display data which has failed to arrive in a timely manner, the zooming operation may 
proceed without the display data, and present the display data whenever it arrives. This may be 
true even if the display data arrives during the zooming process. 

Fade Effects During Zoom 

When zooming in to a node, the contents of the inner structure can begin to be drawn as 
soon as the icon is larger than a few pixels; when the icon is very small, the representation of the 
inner structure is omitted. It is preferred to represent the node with a recognizable icon from a 
very small size up to a reasonable size, and only begin to show the internal structure when the 
icon gets larger than some threshold value. A recognizable icon is easier to understand than a 
minutely drawn diagram. Deferring the drawing of the internal structure until it is large enough 
to be usefixl also improves performance of the computer system, since the number of graphs that 
need to be rendered is limited to those that are visible within the computer display and are large 
enough to be useful. 

In order to make the user interface easily understood and navigated, the transition from 
an icon to a diagram or other representation of the inner structure is made with a gradual fade-in 
effect. This ensures that the user retains a feel for the logical relationships among the objects. 

rnnnecting Links To Internal Elements 

When the user zooms into a node that has one or several Unks, the structure contained 
inside the node is shown in the user interface. The links that connect to the node may continue to 
be shown connected to the outer edge of the node, which is represented as the container of the 
inner structure. In many cases, the link that is shown connecting to the containing node is really 
connected to a specific node in the contained structure. For example, a network diagram may 
show a connection to a building, but when the user interface is zoomed in to show the various 
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computers and other devices in the building, it is preferable to see the connection as going to a 
specific computer. 

If the situation is reviewed in the other direction, a link that connects to an inner node 
inside a structure should, when the user interface is zoomed out to collapse the structure into a 
single icon, is converted to a connection to the icon. 

In the system, as the user zooms in, the icon representing the node is transitioned to the 
containing diagram, preferably with a fade effect. At the same time, the link shown comiecting to 
the node is adjusted to connect to the inner node. 

If the icon is transitioned to the contained structure diagram through a fade effect, the link 
transition is also done with a continuous transition. If the inner structure is displayed without a 
fade effect, suddenly appearing as a replacement for the icon, then the link would undergo a 
similar sudden transition. 

rnnsnlidation of Connections 

It is common in a graph that there may be several links between nodes in two structures 
that are consolidated into two icons at a higher level of the diagram. For example, if there are 
two buildings that each contain several computers, there may be several network links 
connecting pairs of computers in the two buildings. 

When the user zooms out, reducing the two structures to two simple icons, the 
comiections between the several pairs of imier nodes are represented as comiections between the 
higher-level icons. 

hi some cases, it may be preferred to show all the comiections, even on the higher level, 
to give the user a feel for the number of comiections. This results in a number of parallel links. 

hi other cases, it may be preferred to consolidate the large number of comiections into a 
smgle comiection between the two higher-level nodes. Tliis makes the user interface easier to 
read and understand. 

m some cases, the various links at the lower level may represent different types of 
comiections. hi that case, it may be preferable to consolidate links of like type, while still 
showing several links between the higher level nodes, each representing one or several links of a 
specific type. 
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Such propagation and consoUdation of Unks when ascending the containment hierarchy 
has not been employed in a system based on continuous zoom of nested graph structures. 
THpnHnpatinn of Cont ainer TvDC. N^niP and Properties 

When the user interface zooms into an icon and the icon is opened up into a diagram of 
the contained structure, it is of course possible to simply remove the higher level, containing icon 
and show the contained structure on the higher-level background surface. This reflects the 
essence of the contaimnent situation: the contained graph is merely a graph within the larger 
Structure. 

In many cases, however, it is useful to render the container in such a way that the 
contaimnent relationship is visually identified, and the nature and identity of the container are 
obvious. To this end, when an icon is opened to show its contents, it is converted to a contamer 
very much in the style of conventional windowing systems: it has a title bar with the name of the 
container, with an icon in the upper left comer identifying the type of the container. 

In addition, it is often usefiil to show properties of the container. For example, in network 
management applications, it is common to indicate the status of an object by coloring it red, 
orange or yellow. When the icon is expanded into a container, it is of course possible to color the 
entire container, but such a dramatic rendering may be counter-intuitive, since it emphasizes the 
red status for larger containers over smaller ones. Instead, in the appUcants' system, the status of 
the container is usually indicated by coloring the title bar. Other properties may be indicated 
through other icons or colorizations on the container or title bar. 

Although a rectangular container is the most common, it is often very usefixl to draw the 
container as some other geometric shape, such as a triangle or circle. 

Bflpk ffround Maps 

To help the user recognize the type, identity and properties of a container, the system can 
draw a background image when the container is opened. Such a background image can represent 
the opened object in the form of an enlarged version of the icon, a logo, or whatever visual effect 
is considered suitable. The background image may be specified using any type of graphical file, 
including bitmaps, vector files. HTML or other types of graphics. 
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In some cases, it is desirable to use as the background an image that identifies locations. 
This might be a street map or a building floor plan, for example. In these cases, icons in the 
contained structure may be placed on the background map in the correct place. Placements may 
be made manually, through drag-and-drop techniques, or by entering some coordinate that 
identifies a location: latitude and longitude, street address, zip code, phone number, or office or 
cubicle number. 

While certain placement techniques have been implemented in the past, applicants' 
system is the first system to combine this technique of physical placement on a background map 
with the continuous zoom and pan capabiUty. Ms makes the use of such placement considerably 
more usefixl, since it permits the detailed placement and yet retains the larger perspecUve of 
where the whole container is. Tliis is usefixl both when the maps on two different levels of the 
diagram are based on a similar map and when they are different, such as a building's floor plan 
contained within a logical network diagram without physical representation. 

wi ffh Prndnction V}«ings in Graph n{»»g»-am User Interface 

Applicants' system uses advanced rendering techniques with anti-aliasing, tinting, 
translucency and other effects to make the diagram legible and attractive during the continuous 
zooming. This is in contrast to conventional graph diagramming user interfaces that have used 
traditional graphics techniques, which work well enough at a fixed size or at integral zoom 
factors, but they do not render well when the system supports continuous zoom. 

To make clearer the overlay structure of containers on the background, the system uses 
drop shadows to delineate the different layers of the nested diagram. To make this visual effect 
unobtrusive and yet effective, the system uses a translucent shadow with a blurred edge. 
Although these techniques are of course well known in graphics processing systems, but have 
never before been employed.in a user interface designed for managing graph structures. 

Translucency nf Backgrou nd Surfaces 

In some cases, a comiection may pass underneath a container. In classical graph 
rendering user interfaces, the line just disappears under the container and reappears at the other 
end. This make the diagram difficult to read. 
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In the applicants' system, using the advanced rendering technology discussed above, such 
a line is faintly visible through the slightly translucent background of the container. 
Aria ptation to Limited Comp uter Power 

AppUcants' system automatically adapts itself to the observed performance of the 
computer. If the update frame rate during zooming and panning are deemed insufficient, the 
system disables effects such as anti-aliasing, translucency and background maps while there is 
large-scale motion in the display, and re-renders them once the display has stabilized. Although 
on many modem computers, there is ample processing power to render the advanced 
visualization effects in with acceptable performance even during dynamic behavior such as 
zooming, experience shows that responsiveness is critical to a pleasing environment. 

rarefullv Managed Dynamics 

In any graph diagramming user interface, there will be different techniques for 
navigating. Hie user can manually zoom and pan, using the mouse in combination with various 
key sequences on the keyboard: for example, Ctrl+drag up and down might zoom the display, 
while space bar+drag might pan the display. 

It is also common in such user interfaces to provide various forms of automatic 
navigation. For example, double-clicking on an icon usually opens the icon and displays its 
contents; a button on the toolbar steps up one level in the contaimnent hierarchy, collapsing the 
current graph into an icon in another graph. 

In applicants' system, die corresponding operations are done through automatic zooming 

of the diagram in or out. 

To make the behavior of the user interface pleasant and easily understood, the dynamic 
behavior of the visual effects are carefiiUy tuned. For example, when the diagram is 
automatically zoomed in or out, the speed of the effect is gradually increased up to a maximum 
zoom speed, and then gradually decreased down to zero; the entire tiansition is timed to be visual 
but not dizzying. 

To aid in pamiing, the system supports "tossing" the diagram in one direction, by making 
a rapid dragging gesture with a mouse. Tlie diagram glides along and gradually slows to a stop 
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under the effect of simulated friction; if it hits the edge of the large virtual space, it bounces 
back. 

User Rearranf^ement of Layout 

When rendering a graph structure as a diagram of icons and lines, the arrangement of the 
icons and lines on the diagram surface can have a large impact on the clarity and impression of 
the diagram. The system arranges the symbols in different structures by, among other things, 
making abest guess in choosing the most suitable arrangement, depending on the structure of the 
diagram. The system also allows the user to choose another layout mode, or to switch to manual 
mode and arrange the layout by dragging icons on the surface. 

The system can be used in a collaborative environment, where several users view a 
shared database. However, an individual's rearrangement of diagram layouts are considered 
personal, and do not affect other users of the system. To ensure that the user will see the same 
diagram layout regardless of which physical computer he or she uses to view the information, the 
personal layout specifications are stored in the shared database, identified as belonging to the 



user. 



O ppninP or ClosinP Individu al Nodes 

The standard description of a data structure like the one used here, a contaimnent 
hierarchy of nested graphs, emphasizes the fixed hierarchy. This is one reason why conventional 
systems for visualizing graph structures have been based on expUcitly opening and closing 
individual graph windows. In some cases, however, the user may prefer to blur the level of the 
hierarchy when viewing the data structure. 

For example, when viewing a network diagram that has several computers in Chicago 
comiected to several computers in New York, a user may want to see those individual computers 
and the links between them. At the higher level, the diagram shows only an icon for Chicago and 
New York-and between them, several other icons for cities like Detroit and Buffalo. When 
zoomed in to show the individual computers, the containers for Detroit and Buffalo would also 
be visible and take up so much space in the middle of the diagram that the contents of Chicago 
and New York would not be simultaneously visible. Since the user is not interested in Detroit 
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and Buffalo, it would be preferable to either hide those containers, or show them collapsed to 
icons as they were on the higher level diagram. 

Thus, the user may prefer a hybrid diagram that mixes symbols from different levels. 

Applicants' system permits individual containers that are open at one level to be closed to 
icons, at which time the other content of the diagram is rearranged to take advantage of the freed 
space. Further, the system allows a closed icon to be expanded in place into an open container, 
at which time the other content nudges aside to make room for the newly opened container. 

TTspT Restructmi nff of the Hierarchy 

In most cases, the contaimnent hierarchy of a system is imphcitly defined by the 
semantics of the data, or expUcitly defined by a system administrator. Since such a contaimnent 
hierarchy may carry meaning that makes it significant for processing of the data, rearranging the 

hierarchy is not done lightly. 

However, to make the most sense for an individual user, it might be useful to be able to 
rearrange the structure of the diagrams. For example, a network administrator may look at a 
network segment which contains 250 intercomxected computers, all of which are semantically 
meaningful terms. However, the administrator may want to focus the majority of his or her 
attention on the 25 servers nmning business processing, and may not be very interested in the 
225 desktop machines rumiing Windows 98. In a conventional graph diagramming user 
interface, all the less relevant computers clutter up the display, making the important servers or 
other components of interest hard to see. 

While using the system, an individual user may select an arbitrary set of computers, using 
standard desktop metaphors such as dragging a rubber rectangle or Ctrl-clicking with the mouse, 
and group them into an ad hoc container. This container may then be collapsed into an icon. This 
embodiment allows a user to moves the less relevant computers (or other components) out of the 
way. without losing the link to them. 
Filtering 

A graph diagram can easily become cumbersome because of sheer data overload. Since 
graph diagrams typically reflect the physical reality or some other data structure driven by other 
processing, there may be so many objects that the diagram is difficult to read. Further, many of 
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these objects may be irrelevant to a specific user at a particular time. To address this, the system 
provides filtering techniques, which can be used to hide objects in the diagram temporarily based 
■ on their type, status or other property value. 

The system provides a filtering technique based on the inclusion of objects in an arbitrary 
user-defined container. For example, a manager may define a group that contains only those 
systems that are relevant to his or her activities. The diagramming system can be set to show 
only those objects that are included in such an arbitrary grouping. The filtered diagram can then 
be used with the features described herein for the system of the present invention. 
Shortcuts 

A user of the system may want to include a reference to an object in a particular 
container, although the object may be located in another container. For example, in a container 
that includes all the servers that make up a web site, it might make sense to show an icon 
representing a mainframe used by the web servers, even though the mainframe is correctly 

shown in another container. 

The system pemits the addition of references to other objects at any point in a diagram. 
Such icons, which represent the referenced object, may be intercomiected in the diagram and in 
general treated like any regular object. 

rnmhination With Tree Control 

Although the zooming graph display feature described above permits arbifrary navigation 
through a very large structure, there are times when it may be preferable to use other techniques 
for navigation. For example, for a quick jump to another known location, clicking in a 
conventional tree control may be preferable. 

In addition, the nested graph diagrams give an excellent view of the local context, but it 
may be difficult to identify the current location within the larger context. For this reason, the 
graph diagramming display may be supplemented with a "you-are-here" display. 

With applicants' system, both of these needs are met by a tree control that is kept 
synchronized with the graph diagram. As you navigate in the nested graph diagrams, the tree 
controls show where you are; and if you select a node in the tree control, the diagram is 
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automatically navigated to that location. The tree control may be shown or hidden, at the user's 

discretion. 

Vnn- A re-Here Display 

Another useful navigation tool is a small thumbnail map of the entire virtual space, 
indicating the present position with a small rectangle. This "you-are-here" map also permits 
navigation by dragging the rectangle on the map. 

This technique is known in graphics systems, but has never before been applied to a 
continuous zooming and panning graph diagram display such as applicants' system. 

Ap pKnortnn »f HvnerboMc T r«>P TT.er Interface to Network Management 

Unicenter TMD uses a novel application of the known hyperbolic tree visualization 
technique to address the problem of navigating network links or other relationships in a network. 

Since there are so many types of links among the various objects represented in a 
network graph, the hyperboHc tree is extended with a selector that allows the user to specify 
what type of link is to be mcluded in the hyperboUc tree. 

Filtering of nodes based on type, property values or membership in other containers can 

further simpUfy the diagram. 

The status of nodes and links, and other property values such as volume of traffic, is 

represented in the hyperboHc tree in the form of color or icon choice. 

The nodes and links in the hyperbolic tree represent real objects in the network. The 
network management system provides a large number of operations that can be invoked on an 
object, when the objects are represented in standard user interface tools such as regular tree 
controls and Ust boxes. It is an essential feature of applicants' system that those same operations 
are available in the hyperbolic tree as well, presented as items on a context menu, main 
application menu, keyboard sequences or other standard user interface techniques. 

n ynamir and Self-Co nfi y>uring Visu allTatinn Framework 

hi conventional systems, a particular representation is employed by a graphical user 
interface based on knowledge of the data to be represented. For example, a network 
diagramming system has a lot of programming logic referring to the structure of networks in the 
graphics component. 
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For simpler representations, such as tree and Ust browsers, generic visualization tools do 
exist However, even for those tools, the specifications for how the information is to be 
visualized are held either in the graphics code itself or in a database or registry on the machine 

where the visuaUzation is done. 

Both of these techniques are inconvenient, because they make it difficult to reuse the 
graphics tools to visualize new data that resides on a remote computer. Such new data may be 
data that has not been previously considered to be in the domain of the graphics tools, as well as 
data considered to be in the domain of the graphics tools. Applicants' system provides a general 
and dynamically reconfigured visualization tool that takes its specifications for how the data is to 
be represented from a possibly remote data provider, and that provides sophisticated 

visuaUzation. 

Architecture 

THe system relies on a data refrieval infrastructure that permits visualization of such new 

data. 

The system provides objects, sets of objects, associations (relationships or links) between 
the objects or sets of objects, and self-documenting data (e.g. metadata) so that data from 
relational infrastructures can be visualized. As an example, a tuple, such as a single row m a 
relational database, can be viewed as a degenerate object, and that a row set, such as a set of 
rows in a relational database, can be viewed as a set of degenerate objects. TTius, data from 
relational infrastructures can be visualized. 

It should be noted that foreign key relationships among tables in a relational database are 
a form of association contemplated by the present apphcation. 

Although relational systems meet these requirements, many other, more general systems 
meet the requirements as well. In particular, it is common that data is represented as a graph of 
intercomiected objects, which camiot be conveniently or efficiently represented as a table. 

In the preferred embodiment, applicants' system is based on an infrastructure descnbed 
in more detail in Provisional Application Serial Number 60/131,019 filed April 26, 1999 which 
is incorporated herein by reference. 
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The visualization tools in this infrastructure contain a general visualization framework, 
which provide a number of visualization techniques: 

A 2-D graph diagramming tool that provides for navigation of nested and interlinked 
structures through continuous zoom and pan 

A 3-D visuaUzation tool that displays the mfotmation in the fonn of reaUstic or stylized 
3-D enviromnet>ts and provide navigation within the environment. 

A hyperbolic t«e visualization tool that makes it convenient to navigate in very large and 
bushy ^aph structure conventional tree controls. Ust boxes, spreadsheets and property sheets. 

h addition, the visualization ftamewotk supports the construction of visuahzaUon plug- 
ins Although this plug-in architecture may of course be used to build data-specific visualizafon 
tools, that is not the pun>ose of applicants' system; rather, it is intended ^ such plug-ins be 
built in the same way as the general visualization tools provided with the system, configunng 

themselves automatically from data. 

m addition to the regular metadata, which allows the visualization framework to 
dynamically construct property sheets and tables, the architect«e is based on *e data providers 
delivering visualization specifications in the fonn of hints addod to the general metadata. 

These hints may specify, for example, where the icon or 3-0 model for an object .s to be 
found The hint may specify me icon directly, it may specify that a class-level property holds the 
icon for all objects of a certain class, it may specify that an object-level property holds the .con 
for each object, or it may specify that a property holds a set of icons and which one is to be used 
depends on another property (such as status). 

TT,e hints may be very detailed For example, for a successful 3-D visualization, the htnts 
may specify several extemal models to be used at different levels of detail, as well as an inten«l 
model, a floor texture, and speciaUzed characteristics such as fte radius used for colhston 

detection. , 
The hints may specify one or more types of associations used to represent the 

containment hierarchy used in trees, diagrams and 3-D views, and one or more types of 
associations that can be shown as links in those diagrams. 
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And finally, fte hine may specify menu items that are to be displayed on context menus 
for each class of object, and the path to the method that implements each menu item. 

With this infrastructure and these extensive hints supplied by the information provrder, 
fl,e visualization framework can represent any data that meets these very broad requirements m a 
number of very sophisticated ways. 

Variation: Ex* *"-""' Provider 

to some instances, an infomtation provider may not have die visualization hints that are 
needed for the proper wotldngs of the visualization frameworlc. and it may no. be convement. 
permittedorpossibletoextendtheproviderwithvisualizationhmts. 

to such instances, the frMtework pemuts the specification of an external provder of 
visualization hints for an information provider. 

under apphcants' system, the pe^on responsible for providing «.e dam can provrde the 
visualization hints and place them at some convenient location, near the data provda or 
elsewhere, but without having to distribute them to thousands of systems. 

E. Neugent 

-„f, 1. rh"-- — Neo. II Np^^ nrlr ^mtyis Services T. Remote 

Neural network technology is a powerful tool for solving many types of problems. The 
basic mathematics ofneural network technology are well understood. 

Apphcants- system provides a convenient way of com,ecting neural network techndogy 
.0 common appUcations. regaMless of the programming language used, and regardless of fte 
location of the user interfec the data source or the processing resources r^mred by the neural 
network. 

Architecture . 

The system of the present applicaUon can be configured with a neural network processmg 
service that is connected to a remote access mechanism. The remote access mecham^sm can be 
any object request broker, such as CORBA or Microsoft's DCOM. Preferably, the .nftastructure 
described above and in Provisional Application Serial No. 60/131.019 is utilized. 
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The neural network service provider is configured as a class. When using the services, the 
client application creates an instance of the neural network class. This instance holds the 
properties that define the task of the neural network, and also holds the model that the neural 
network generates after training. The instance is persisted by the neurd netwotk provider m 
some type of data store. The provider can use any conventional persistence mechamsm, 
including SQL and a regular file system, to the preferred embodiment, the provider uses fte 
object database of the preferred inftastrucure. 

The application performs these tasks: 

Instantiate a neural network object which automatically persists its information; 
Depending on what type of problem is to be solved, specify some small number of 

parameters; . . 

Tell the neural network object where its training data is, and tell it to start trammg; and 

Tell the neural netwotk object where its consulting data is, and consult it. 

Neural networks can be used to do different types of analysis, and to address these 
different needs. In one embodiment, the syst™ uses three ditfaent classes of neural networks: 

Value prediction 

Event prediction 

Cluster analysis 

The different types of neural network objects require different parameters. For example, 
value prediction requires specification of which fields are to be predicted (the "outputsn. 

Some parameters are optional. For example, value prediction normally assumes that all 
fields that are not outputs are inputs, bu, fte application program may optionally list the mpu. 
fields specifically, implying that those that are left out are to be ignored. Wherever posstble, all 
properties are optional, with reasonable values assumed. 

One reason for using a neural network is providing the data for training as well as 
consultation. Since the neural network features can be used for many diverse functions, a way to 
increase the efficiency of the neural network technology is to permit an application program to 
specify the pafl. to the data; so that the neural network retrieves the data when it needs tt. usmg 
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the data retrieval infrastructure it is connected to. This removes the need to move data to the 

location of the neural network. 

It is common in modem architectures to have the client with the user interface on a 
system separate from the database server that holds the data. But since fraining a neural network 
may be demanding of computing resources, it is preferred that the architecture allows efficient 
invocation of the neural network when it is placed on a third system, separate from either client 
or data server. And in this case, it is also preferred that the neural network refrieves the data 
directly from the data server, without requiring that it is passed through the client. 

Similarly, after the neural network object has been frained, and the created model has 
been persistently stored, the application consults the neural network in the same way: it specifies 
the location of the data, and the target path for placing the results, and asks for a consultation. 

In some cases, however, the consultation data may exist in the client application already, 
after having been entered by the user. To support such situations, the system permits consultation 
from a collection of data objects passed in as arguments. 

Neural networks may also be used to predict events, hi this case, the data source is 
presented the same way as in the value prediction case, but the result is an event, not a set of 
predicted values. The system uses the infrastructure to send the predicted events, using the 
standard event propagation mechanism. 

8. rnm pliance Witb Fv^miner Guide "i^« For Compufer Related Inventions 

In regard to its practical application, the present invention makes a substantial 
confribution to and advancement of the practical industrial arts in that it allows the user to use a 
visualization workstation to monitor and confrol remote portions of a networked computer 
system, using a real world interface while also providing two dimensional graphical displays and 
other tools. It allows comprehensive management of all resources on the network. Views and 
data relating to a specific business interest of particular concern to a user may be selected for 
viewing. The present invention is user customizable. Finally, it is generally applicable and 
extendable to any equipment or system with computing and agent communication capability. 

The present invention does not fit within any of the per se nonstatutory subject matters 
categories: it is not fimctional descriptive material such as data sfructures or a computer program 
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listing, is not nonfunctional descriptive material such as various literary copyrightable works, 
and is not a natural phenomena in the realm of pure science. 

The present invention comprises an inventive combination of software and hardware. 
Specifically, this application comprises a Virtual Reality (VR) Workstation(s) and Object 
Repository Server communicating and controlling the enterprise client-server system via a 
TCP/IP or other comiections. The VR Workstation requires an advanced processor of at least an 
Intel Pentium® 586 processor, a 3-D accelerated video board with OpenGL support, and at least 
32 MB of Random Access Memory (RAM). The software portion of the preferred embodiment 
uses Windows NT as an operating system in both the VR Workstation and Object Repository 
Server The Object Repository includes a database for maintaining the status of the enterprise 
cUent-server system. The present invention thus is a product (machine or manufacture) for 
performing a process and is thus statutory. 

The present invention, to the extent that it comprises a series of steps to be performed on 
a computer, is a process that manipulates data representing physical objects (e.g., inventory if 
selected on the business interest) and activities on the networked equipment being monitored to 
achieve the practical application discussed above. The inventive process also performs 
independent physical acts after computer processing by presenting practical views to the user on 
the visualization station monitor. The inventive process does not merely manipulate data without 
any practical application. Thus also as a process the present invention is statutory. 

The foregoing inventive system and apparatus has been described generally and with 
reference to preferred and other embodiments. THose skilled in the art, upon reading of the 
specification, will understand that there are equivalent alterations, modifications and 
embodiments including systems that monitor, control, administer, and manage systems that may 
not be labeled "networked computer systems" but which substantively are networked computer 
systems. The present invention includes systems to administer all networked computer systems, 
however labeled, and includes all such equivalent alterations and modifications. 
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